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ter  training  on  computer-aided  design  (CAD)  systems  traditional¬ 
ly  has  been  expensive  and  ineffective.  In  addition,  rapid  changes  in 
CAD  software  demand  frequent  update  instruction  to  be  able  to 
take  full  advantage  of  the  system's  capabilities.  As  the  U.S.  Army 
Corps  of  Engineers  begins  using  CAD  technology  to  an  increasing 
degree,  there  is  a  great  need  for  low-cost,  effective  teaching  programs. 

Embedded  instruction  programs  represent  a  promising  answer  to 
this  need.  This  technology  involves  the  incorporation  of  tutorial 
programs  directly  into  the  software  being  taught,  with  the  software 
used  to  drive  the  training  session.  Such  a  leaching  method  allows 
users  to  participate  in  self-paced  study  on  the  system  they  will  be 
using  in  actual  day-to-day  operations. 

The  U.S.  Army  Construction  Engineering  Research  Laboratory 
(USA-CERL)  has  developed  and  tested  an  embedded  instruction 
program  for  teaching  CAD.  The  program  was  demonstjaladjn  a  field 
test  funded  under  the  Technology  Transfer  Test  Bedf^Bt  program. 
The  field  test  results  indicete  that  architects  and  engineers  profit 
from  online  instruction  embedded  in  the  CAD  system  that  they  are 
learning.  There  is  wide  veriability  in  the  time  spent  studying  the 
lessons  as  well  fcs  In  learning  strategies  employed  by  different 
students.  Follow-up  questionnaires  revealed  a  preference  for  this 
type  of  instruction  over  traditional  methods.  ( ^ 

The  program  developed  by  USA-CERL  has  been  transferred  to 
industry  through  a  Cooperative  Research  and  Development  Agree¬ 
ment  (CRDA)  in  accordance  with  the  Federal  Technology  Transfer 
Act  of  1986.  The  agreement  is  with  Electronic  Courseware  Systems, 
Inc.,  which  after  refining  the  program,  has  named  it  'Teaching 
Assistant  for  AutoCAD"  and  placed  it  on  the  commercial  market. 
This  version  of  the  Teaching  Assistant  is  designed  for  the  AutoCAD 
microcomputer  software;  a  second  version  for  Intergraph's  Micro- 
station  CAD  system  is  currently  under  development. 
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TECHNOLOGY  TRANSFER  TEST  BED  PROGRAM 
FINDINGS  AND  RECOMMENDATIONS  OF  TEST/DEMONSTRATION 

WORK  UNIT  NO./TITLE  OF  TEST:  T^B  Project,  "Computer-Aided  Instruction 
(AutoCAD)" 

PERFORMING  LABORATORY:  USA-CERL 

PRODUCT/SYSTEM:  Teaching  Assistant  for  AutoCAD 

TEST  SITES:  Huntsville  Division,  New  England  Division,  Baltimore  District 

DESCRIPTION/OBJECTIVE  OF  TEST/DEMONSTRATION: 

The  objective  was  to  field-test  a  computer-based  educational  program  that  uses  the 
embedded  instruction  concept  to  teach  architects  and  engineers  (A/Es)  how  to  use  com¬ 
puter-aided  drafting  (CAD)  software.  The  program  is  intended  to  serve  as  an  alternative 
to  traditional  forms  of  training,  which  can  be  costly  and  time-consuming.  In  addition, 
embedded  instruction  provides  for  self-paced  learning  to  optimize  time  spent  on  les¬ 
sons.  The  goals  of  the  test  were  to  validate  the  program's  effectiveness  in  training  users 
and  to  determine  its  responsiveness  to  professionals  with  different  learning  patterns. 

RESULTS  OF  TEST/DEMONSTRATION: 

The  84  test  participants  provided  feedback  to  document  their  experience  with  the  Teach¬ 
ing  Assistant.  A  questionnaire  was  also  completed  to  measure  attitude.  Results  were 
analyzed  and  showed  that,  in  general,  A/Es  benefit  from  online  instruction  on  the  CAD 
program  they  wish  to  learn.  The  wide  variation  in  time  spent  on  the  lessons  and  in  the 
learning  strategies  used  reinforce  the  goals  of  embedded  instruction— self-paced,  self- 
serving  education.  Results  further  suggested  that  the  program  is  flexible  enough  to  en¬ 
sure  responsiveness  to  different  levels  of  user  expertise.  Most  participants  indicated  a 
preference  for  this  form  of  training  over  conventional  classroom  methods. 

RECOMMENDATIONS  FOR  PRODUCT/SYSTEM: 

O 

The  successful  outcome  of  this  T  B  demonstration  has  resulted  in  a  Cooperative  Re¬ 
search  and  Development  Agreement  (CRDA)  between  USA-CERL  and  a  private  company, 
Electronic  Courseware,  Inc.  (ECS).  This  CRDA  was  possible  through  the  Federal  Tech¬ 
nology  Transfer  Act  of  1986  and  Public  Law  99-502.  ECS  has  refined  the  prototype  sys¬ 
tem  and  is  marketing  it  under  the  tradename  "Teaching  Assistant  for  AutoCAD." 

Embedded  instruction  has  a  very  far-reaching  potential  for  application  in  teaching  other 
types  of  computer  programs.  In  addition  to  the  advantages  of  self-paced  learning  and 
reinforcement,  this  type  of  instruction  offers  a  lower  cost  alternative  to  classroom 
training.  There  is  a  huge  potential  savings  in  terms  of  employee  travel  expenses,  tuition/ 
materials,  and  the  cost  of  retraining  or  refresher  courses.  One  set  of  the  Teaching 
Assistant  software  can  be  shared  among  several  users;  in  addition,  training  time  is  not 
dependent  on  an  outside  organization's  schedule,  but  can  be  incorporated  into  the  daily 
workflow.  The  Army  should  explore  other  applications  for  this  technology  to  ensure 
effective  training  on  software  and  avoid  the  cost  of  offsite  instruction  when  practical. 


UNCLASSIFIED 

security  Classification  of  this  page1 


la  REPORT  SECURITY  CLASSIFICATION 

Unclassified 


2a  SECURITY  CLASSIFICATION  AUTHORITY 


2b  OECLASSIFICATION/ DOWNGRADING  SCHEDULE 


4  PERFORMING  ORGANIZATION  REPORT  NUM8ER(S) 

USA-CERL  TR  P-89/03 


REPORT  DOCUMENTATION  PAGE 


lb  RESTRICTIVE  MARKINGS 


Form  Approved 
OMB  No  0704  01  SB 
Eip  Date  lun  JO  1986 


6 a  NAME  OF  PERFORMING  ORGANIZATION 
U.S.  Army  Construction  Engr 


6c  address  (Cl ty.  State  and  ZIP  Code) 
P.0.  Box  4005 
Champaign,  IL  61820-1305 


Sa.  NAME  OF  FUNDING  -  SPONSORING 
ORGANIZATION 

HQUSACE 


8c.  ADDRESS  (City.  Stitt,  and  ZIP  Code) 

20  Massachusetts  Ave.,  N.W. 
Washington,  D.C.  20314-1000 


6b  OFFICE  SYMBOL 
(if  applicable) 

CECER-FS 


3  DISTRIBUTION/AVAILABILITY  OF  REPORT 

Approved  for  public  release;  distribution 
is  unlimited. 


5  MONITORING  ORGANIZATION  REPORT  NUM8ER(S) 


7a  NAME  OF  MONITORING  ORGANIZATION 


7b  ADDRESS  (City,  State,  and  ZIP  Code) 


3b  OFFICE  SYMBOL  9  PROCUREMENT  INSTRUMENT  IDENTIFICATION  NUMBER 
(if  applicable ) 

CEEC-ED  T^B  "Computer-Aided  Instruction  (AutoCAD)" 


10  SOURCE  OF  FUNDING  NUMBERS 


PROGRAM 
ELEMENT  NO 


PROJECT 

TASK 

NO 

NO 

WORK  UNIT 
ACCESSION  NO 


11  title  (include  Security  Classification) 

Intelligent  Embedded  Instruction  for  Computer-Aided  Design  (CAD)  Systems  (U) 


12  PERSONAL  AUThOR(S) 

Shaw,  Doris  Smith;  Golish*  L.  Michael;  Johnson,  Robert  L. _ . 


Ub  TIME  COVERED 
FROM _  TO 


14  DATE  OF  REPORT  (Year.  Month.  Day) 

1988,  October  _ 


13a  TYPE  OF  REPORT 

final 


16  supplementary  notation 

Copies  are  available  from  the  National  Technical  Information  Service 

Springfield,  VA  22060 


FIELD 

GROUP 

05 

06 

COSATI  COOES 


SUBGROUP 


18  SUBJECT  TERMS  ( Continue  on  reverie  if  necessary  and  identify  by  block  number) 
embedded  instruction  Teaching  Assistant 

computer-aided  design 

training  _ 


19  ABSTRACT  (Continue  on  reverse  if  necessary  and  identify  by  block  number) 

User  training  on  computer-aided  design  (CAD)  systems  traditionally  has  been 
expensive  and  ineffective.  In  addition,  rapid  changes  in  CAD  software  demand  frequent 
update  instruction  to  be  able  to  take  full  advantage  of  the  system's  capabilities.  As  the 
U.S.  Army  Corps  of  Engineers  begins  using  CAD  technology  to  an  increasing  degree, 
there  is  a  great  need  for  low-cost,  effective  teaching  programs. 

Embedded  instruction  programs  represent  a  promising  answer  to  this  need.  This 
technology  involves  the  incorporation  of  tutorial  programs  directly  into  the  software 
being  taught,  with  the  software  used  to  drive  the  training  session.  Such  a  teaching 
method  allows  users  to  participate  in  self-paced  study  on  the  system  they  will  be  using  in 
actual  day-to-day  operations. 

_ (Cont ’d) 


20  distribution  .  availability  of  abstract  21  abstract  security  classification 

□  unclassifieD'UNlimited  09  same  as  ppf  O  one  users  Unclassified  _ 


•r  MO'VIDI  ial  |22b  TELEPHONE  (Include  Area  Code)  22c  OFFICE  SYMBOL 

(217)  352-6511  x  389  CECER-IMT _ 


DO  FORM  1473,  84  MAR  83  APR  edition  may  be  used  unM  exhausted  SECURITY  Classification  QF  this  pAG£_ 

AH  other  editions  ate  obsolete 

UNCLASSIFIED 


I 


UNCLASSIFIED 


Block  19.  (Cont'd) 

The  U.S.  Army  Construction  Engineering  Research  Laboratory  (USA-CERL)  has 
developed  and  tested  an  embedded  instruction  program  for  teaching  CAD.  The  program 
was  demonstrated  in  a  field  test  funded  under  the  Technology  Transfer  Test  Bed  (T3B) 
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ity  in  the  time  spent  studying  the  lessons  as  well  as  in  learning  strategies  employed  by 
different  students.  Follow-up  questionnaires  revealed  a  preference  for  this  type  of 
instruction  over  traditional  methods. 

The  program  developed  by  USA-CERL  has  been  transferred  to  industry  through  a 
Cooperative  Research  and  Development  Agreement  (CRDA)  in  accordance  with  the  Fed¬ 
eral  Technology  Transfer  Act  of  1986.  The  agreement  is  with  Electronic  Courseware 
Systems,  Inc.,  which  after  refining  the  program,  has  named  it  "Teaching  Assistant  for 
AutoCAD"  and  placed  it  on  the  commercial  market.  This  version  of  the  Teaching  Assist¬ 
ant  is  designed  for  the  AutoCAD  microcomputer  software;  a  second  version  for  Inter¬ 
graph's  Microstation  CAD  system  is  currently  under  development. 
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This  project  was  demonstrated  as  part  of  the  Technology  Transfer  Test  Bed  (T3B) 
program  under  the  T38  Work  Unit  entitled  "Computer-Aided  Instruction  (AutoCAD)." 
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(HQUSACE),  under  Project  4A162731AT41,  "Military  Facilities  Engineering  Technology"; 
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Technical  Monitor  was  Donald  Dressier,  CEEC-ED. 
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Construction  Engineering  Research  Laboratory  (USA-CERL).  Dr.  Michael  O'Connor  is 
Chief  of  USA-CERL-FS.  The  USA-CERL  technical  editor  was  Dana  Finney,  Information 
Management  Office. 
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INTELLIGENT  EMBEDDED  INSTRUCTION  FOR 
COMPUTER-AIDED  DESIGN  (CAD)  SYSTEMS 


1  INTRODUCTION 


Background 

To  ensure  continued  productivity  and  quality  of  work,  the  U.S.  Army  Corps  of  Engi¬ 
neers  (USACE)  is  increasingly  switching  to  automated  methods  in  its  daily  operations. 
One  technology  with  a  high  potential  benefit  to  USACE  is  computer-aided  design  (CAD), 
which  can  improve  the  cost-effectiveness  and  accuracy  of  design  drawings  for  facilities 
requiring  a  great  deal  of  unit  repetition,  as  is  common  with  most  military  construction. 
In  addition,  CAD  offers  the  advantage  of  storing  "standard"  designs  online  for  use  by 
architects/engineers  (A/Es)  throughout  USACE. 

It  has  become  apparent  that  many  current  CAD  systems  are  not  being  used  to  their 
full  potential,  making  them  less  cost-effective  than  intended.  Part  of  the  reason  for  this 
deficit  is  that  training  programs  for  the  CAD  software  have  been  ineffective  and 
costly.  Customary  classroom  methods  of  training  are  not  suitable  for  designers:  time 
schedules  and  individual  needs  vary  too  widely  to  structure  a  course  meeting  everyone's 
needs.  While  individual  tutoring  might  prove  more  effective  from  a  learning  standpoint, 
it  would  be  too  expensive  for  widespread  use  within  USACE.  Adding  to  the  problem  is 
that  new  developments  in  computer  software  to  increase  system  power  occur  almost 
daily,  but  these  advances  require  constant  updating  of  skills.  This  requirement  allows  lit¬ 
tle  complacency  in  the  field;  cost-effective,  continuing  training  of  A/Es  in  the  use  of 
automated  drafting  and  design  tools  is  essential. 

Embedded  instruction  offers  a  promising  alternative  to  classroom  training.  This 
technology  involves  "embedding"  a  tutorial  program  directly  into  the  software  program 
being  learned,  using  the  system  itself  as  a  delivery  medium  for  the  lessons.  CAD  systems 
are  especially  well  suited  to  embedded  instruction.  Users  can  learn  at  their  own  pace  on 
the  actual  software  they  will  be  using  in  daily  work. 

Training  programs  that  use  embedded  instruction  could  both  improve  users'  effec¬ 
tiveness  on  USACE  CAD  systems  and  lower  the  cost  of  employee  training.  To  be  of  most 
benefit  to  USACE,  the  embedded  instruction  program  must  be  "intelligent"  enough  to 
respond  to  the  trainees'  individual  learning  styles  and  objectives. 

The  U.S.  Army  Construction  Engineering  Research  Laboratory  (USA-CERL)  has 
developed  a  training  program  for  AutoCAD  based  on  the  embedded  instruction  concept. 
To  learn  if  this  training  program  would  meet  the  needs  of  USACE  AutoCAD  users,  a  field 
test  was  funded  under  the  Technology  Transfer  Test  Bed  (T3B)  program.  T3B  is  a  USACE 
initiative  to  ensure  that  research  and  development  are  responsive  to  the  customer's  needs 
as  determined  through  site  demonstrations  and  formal  user  groups. 


Purpose 

The  purpose  of  this  research  was  to  develop  and  field-test  a  computer-based  educa¬ 
tional  system  using  embedded  instruction  technology  to  provide  USACE  with  a  low-cost, 
effective  alternative  to  traditional  forms  of  training.  The  program  must  be  "intelligent" 
for  responsiveness  in  training  professional  A/Es. 
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Approach 

The  approach  involved  first  studying  A/Es  in  the  field  as  they  learned  to  use  a  CAD 
system.  A  simple  computer-based  learning  environment  was  modeled  as  an  observation 
tool.  Chapter  2  summarizes  the  educational  requirements  used  to  develop  this  model. 

Computer-based  instruction  was  then  developed  within  a  widely  used  CAD  system. 
(See  Chapter  2  for  a  complete  description  of  the  training  program.)  The  target  audience 
was  design  professionals  and  technicians  within  USACE.  Lessons  covered  the  materials 
commonly  taught  in  introductory  CAD  courses. 

After  the  lessons  were  completed,  they  were  officially  field-tested  at  three  USACE 
sites—Huntsville  Division,  New  England  Division,  and  Baltimore  District.  The  test  sub¬ 
jects  were  asked  to  supply  personal  information  about  experience,  time  spent  on  the  les¬ 
sons,  and  other  relevant  comments.  They  were  tested  for  subject  matter  mastery  and 
then  administered  an  attitude  questionnaire.  Chapter  3  includes  the  findings  of  this  test. 


Mode  of  Technology  Transfer 

In  accordance  with  the  Federal  Technology  Transfer  Act  of  1986  and  Public  Law 
99-502,  the  program  described  in  this  report  has  been  transferred  through  a  Cooperative 
Research  and  Development  Agreement  (CRDA)  between  the  U.S.  Army  Construction 
Engineering  Research  Laboratory  (USA-CERL)  and  Electronic  Courseware  Systems 
(ECS),  Inc.  ECS  refined  the  prototype  for  teaching  AutoCAD,  and  it  is  now  available  on 
the  commercial  market  under  the  trademark  "Teaching  Assistant  for  AutoCAD."  A  sec¬ 
ond  version  of  the  Teaching  Assistant  is  being  developed  for  the  Intergraph  Microstation; 
Intergraph  is  the  commercial  product  selected  for  the  Corps-wide  Computer-Aided 
Design  and  Drafting  (CADD)  Buy.  The  same  concepts  developed  in  this  research  could  be 
adapted  for  a  wide  range  of  applications  within  USACE  and  the  private  industry. 


2  DEVELOPMENT  AND  TESTING 


Selection  of  an  Educational  Method 

Several  factors  pointed  to  computer-based  instruction  (CBI)  as  an  educational  ap¬ 
proach  for  training  A/Es  in  CAD.  Among  these  factors  are  cost  and  applicability  to  the 
type  of  learning  situation  required  by  the  target  audience.  Embedded  CBI,  as  explained 
below,  offers  even  more  benefits  because  of  the  characteristics  inherent  in  the  subject 
matter. 


Computer-Based  Instruction 

Traditional  classroom  methods  of  instruction  often  prove  to  be  inefficient.  They 
usually  assume  that  most  students  have  similar  entry-level  skills  and  similar  objectives. 
Another  assumption  is  that  everyone  will  learn  at  comparable  rates  and  that  all  have  the 
same  block  of  time  available  for  the  class,  lisers  who  do  not  meet  these  prerequisites 
may  not  be  able  to  make  good  use  of  their  time.  Because  CAD  is  a  relatively  new  tech¬ 
nology,  design  professionals  vary  greatly  in  'heir  needs  and  objectives. 

CBI  resolves  the  problems  of  classroom  methods  by  offering  the  advantage  of  self- 
paced  learning,  which  can  be  done  at  users'  convenience  and  according  to  their  individual 
learning  preferences.  This  benefit  is  extremely  valuable  for  adult  professionals  whose 
needs  may  be  highly  specialized  and  whose  schedules  require  flexibility.  Learners  may 
pursue  a  subject  in  depth  or  choose  to  skim  the  material  for  quick  review.  The  increased 
efficiency  of  training  time  could  be  a  great  cost  savings.  Also,  the  classroom  environ¬ 
ment  produces  social  pressure  for  perfection  which  can  limit  experimentation  by  the 
learner. 

CBI  could  still  be  used  in  a  classroom  situation  to  advantage;  teachers  could  let  the 
more  independent  students  work  without  intervention,  but  provide  help  to  those  needing 
more  support.  However,  for  USACE's  needs,  walk-in  training  is  the  more  promising 
environment.  Walk-in  training  offers  the  flexibility  in  time  and  scheduling  that  many 
professionals  need. 

Embedded  instruction  is  a  tutorial  computer  program  that  is  installed  within  the 
actual  computer  system  to  be  learned.  It  can  be  very  effective  in  avoiding  the  problems 
that  occur  when  users  learn  on  one  system  and  then  attempt  to  transfer  their  knowledge 
to  different  equipment.  With  embedded  instruction,  the  situations  encountered  in  learn¬ 
ing  are  the  same  ones  that  can  be  expected  in  actual  use,  so  that  all  experimentation  is 
educational— and  encouraged.  The  lessons  in  these  programs  often  are  sequenced  accord¬ 
ing  to  difficulty  or  classification,  perhaps  not  offering  all  options  at  first,  but  leading  to 
an  understanding  of  the  complete  program  by  the  end  of  the  tutorial.  The  student  can 
learn  to  use  shortcuts  and  other  aids  provided  by  the  program  and  can  compare  different 
methods  for  solving  problems. 

Adult  designers  tend  to  develop  their  own  personal  ways  of  using  CAD  software 
which  can  optimize  a  system's  use.  This  ability  has  been  referred  to  as  "making  the  com¬ 
puter  an  extension  of  oneself"  and  is  based  on  a  conceptual  understanding  of  the  way 
CAD  programs  operate.  To  promote  this  level  of  understanding,  instruction  for  com¬ 
puterized  design  must  allow  for  experimentation  to  help  users  begin  to  conceptualize  the 
CAD  environment.  Such  an  approach  is  in  contrast  to  teaching  methods  that  are  just  a 
step-by-step  presentation  of  procedures. 
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The  Instructional  Program 

Effective  computerized  instruction  is  based  on  knowledge  of  three  areas:  the  sub¬ 
ject,  the  educational  media,  and  the  stages  in  de/elopment  of  expertise  in  the  learner. 
Since  computer-based  instruction  attempts  to  reduce  the  need  for  human  intervention  in 
training,  a  thorough  analysis  of  the  learning  process  is  necessary.  The  usual  accom¬ 
modation  of  teacher  to  student  must  be  predicted  and  simulated  as  closely  as  possible. 

Design  has  traditionally  been  considered  a  product  of  the  imagination,  not  subject 
to  computerization.  Therefore,  development  of  an  instructional  program  involved  not 
only  procedural  knowledge  in  learning  CAD,  but  also  the  concept  of  designing  with  the 
help  of  a  computer.  A  case-study  approach  was  used  to  investigate  patterns  of  thinking 
as  A/E s  learned  to  use  AutoCAD.1  This  system  was  chosen  because  of  its  open  design 
which  offered  the  possibility  of  embedding  instruction  and  also  because  most  USACE  Dis¬ 
trict  and  Division  Offices,  as  well  as  a  large  number  of  commercial  A/Es,  use  AutoCAD 
for  some  of  their  work. 

From  initial  observations,  the  following  requirements  for  the  computer-based  les¬ 
sons  were  established: 

1.  The  course  content  would  be  represented  better  by  a  concept  specification  hier¬ 
archy  than  by  a  traditional  task  analysis  of  skills.2  Different  designers  employed  differ¬ 
ent  procedural  approaches. 

2.  If  A/Es  could  learn  to  envision  the  computer  as  an  extension  of  their  own  design 
processes,  they  might  be  more  inclined  to  use  it.  There  was  little  acceptance  for  the 
idea  of  changing  design  practices  to  comply  with  constraints  set  by  the  computer. 

3.  It  would  be  necessary  to  allow  the  designer  time  and  space  for  individualized 
experimentation.  The  study  showed  wide  variations  in  such  needs. 

4.  In  the  interest  of  efficiency,  however,  some  structure  would  be  needed  in  the 
presentation.  The  first  lessons  should  be  aimed  at  developing  a  general  familiarity;  the 
next  few  lessons  should  help  the  user  grasp  the  concept  of  the  software's  power;  and  the 
last  ones  should  impart  more  detailed  knowledge. 

Based  on  these  observations,  conceptual  difficulties  were  predicted  and  six  lessons 
were  prepared  that  were  aimed  at  preventing  error  pattern  formation.  The  lessons  were 
programmed  in  AUTOLISP,  with  AutoCAD  graphics  to  create  the  screen  displays. 

During  a  lesson,  users  are  monitored  and  provided  with  feedback  if  they  commit 
expected  errors  or  fail  to  practice  a  command.  Extra  help  is  offered  from  some  lesson 
screens  when  the  student  types  the  letter  X.  This  help  is  context-sensitive;  that  is,  the 
same  keypress  will  result  in  different  help  screens  from  different  lesson  screens.  The 
first  two  lessons  provide  a  general  framework;  the  next  two  suggest  different  design  ap¬ 
plications;  and  the  last  two  address  details  of  editing  and  documentation. 


‘D.  Shaw,  "Case  Studies  in  Architectural  CADD  Education,"  Architectural  Education , 
Research,  and  Practice  in  the  Next  Decade,  ACADIA  Workshop  '86  Proceedings,  J. 
Turner  (Ed.)  (University  of  Houston,  October  1986). 

2M.  Birenbaum,  and  D.  Shaw,  "Task  Specification  Chart:  A  Key  to  a  Better  Under¬ 
standing  of  Test  Results,"  Journal  of  Educational  Measurement,  Vol  22  (1985),  pp  219- 
230. 
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The  commands  CONTINUE,  BACK,  and  INDEX  were  added  to  the  AutoCAD  screen 
menu  and  are  always  active.  In  addition,  typing  the  letter  B  at  the  Command  prompt 
produces  a  blank  screen  for  practice.  Example  lesson  screens  are  shown  in  Figures  1 
through  4. 


Figure  1.  Context-sensitive  help.  If  students  do  not  understand  about  base  points  and 
displacement  points,  they  can  type  X  to  receive  context-sensitive  help. 


Th«  base  point  or  displacement  is  a 
point  that  refers  to  the  object  to  be 
moved  or  copied  to  a  new  location. 


Q 


second 


displacement  point  — 
\-base  point 


The  second  displacement  point  is  a 
correspond i ng  point  on  the  object 
in  its  new  or  cop i ed  pos i t i on . 


_ Select  CONTINUE  to  return  to  the  lesson. _ ^ 


Figure  2.  Help  example.  When  help  is  requested,  more  detail  is  offered.  Often  the  help 
includes  graphics  or  animation. 


Figure  3.  Error  feedback.  If  a  common  error  is  made  (e.g.,  pressing  the  Return  key 
rather  than  the  Pick  button),  this  message  attempts  an  explanation. 


r 

The  display 

\ 

you  just  saw  offered  an 

oppor  tun i ty 

to  try  an  AutoCAD  command. 

Select  BOCK 

to  return  to  the  page  and 

experiment. 

V 

Select  CONTI UE  to  go  on. 

j 

Figure  4.  Practice  opportunity.  This  display  is  seen  if  the  student  fails  to  try  the  exer¬ 
cise  when  he/she  is  directed  to  practice. 
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Field  Test 

Three  official  (i.e.,  funded)  test  sites  and  several  volunteer  sites  agreed  to  solicit 
subjects  to  work  through  the  lessons,  answer  a  questionnaire,  and  do  a  test  drawing.  The 
sites  funded  under  T3B  were  three  USACE  Offices— Huntsville  Division,  New  England 
Division,  and  Baltimore  District.  No  written  documentation  was  provided  to  accompany 
the  lessons.  Each  test  site  named  a  test  coordinator  who  was  responsible  for  installing 
the  program,  helping  the  learners  start,  and  being  available  for  questions  at  appointed 
hours.  The  objective  was  to  conduct  the  testing  through  the  site  coordinators  rather  than 
through  an  outside  project  director.  This  arrangement  was  predicted  to  give  more  realis¬ 
tic  data,  even  though  there  might  be  incomplete  records  and  unexpected  differences 
between  the  test  sites  (see  Chapter  3).  In  addition,  this  configuration  was  the  most  eco¬ 
nomical  way  to  use  the  programs  and  was  expected  to  help  reveal  any  weaknesses  in  the 
instruction. 

The  three  official  sites  and  two  of  the  volunteer  groups  sent  their  project  coordina¬ 
tors  to  USA-CERL  for  one  day  of  training  in  administration  of  the  research.  Only  one  of 
the  coordinators  was  experienced  with  AutoCAD;  one  had  no  previous  microcomputer 
experience.  The  coordinators  were  asked  to  give  careful  instruction  to  beginners  on  how 
to  operate  the  lessons  and  the  input  device  in  use  at  the  workstation.  An  additional  test 
was  conducted  in  a  classroom  situation  with  a  teacher  present  for  the  first  4  hr  of 
instruction.  This  test  was  intended  to  assess  the  program's  performance  in  a  formal 
classroom  environment.  Six  students  completed  the  class. 

All  data  were  provided  anonymously.  The  test  director  knew  only  the  numbers  of 
test  subjects,  and  the  site  coordinators  did  not  know  the  test  or  questionnaire  results. 
Demographic  information  collected  included  age,  sex,  computer/CAD  experience,  and 
educational  background.  (Not  all  participants  provided  these  data.)  Each  subject  was 
asked  to  record  time  spent  in  each  lesson.  Appendix  A  contains  blank  subject  informa¬ 
tion  forms. 

The  final  test  drawing  was  a  USACE  castle  for  which  precise  specifications  were 
given  (Figure  5).  Students  were  asked  to  record  the  time  spent  on  the  drawing.  Items 
graded  were:  drawing  location,  intersections,  circles  and  arcs,  angles  of  lines,  layering 
(linetype  and  color),  and  block  creation  and  insertion.  A  program  written  in  AUTOLISP 
examined  the  values  in  the  drawing  file  and  calculated  the  drawing  score.  Appendix  B  is 
the  drawing  test  presented  to  subjects. 


Research  Questions 

In  addition  to  performance  rating,  this  study  was  designed  to  assess  attitudes 
regarding  computer  use,  CAD,  computer-aided  instruction,  perceived  knowledge  about 
CAD,  and  the  lessons  themselves.  A  questionnaire  was  designed  to  measure  all  of  these 
factors.  There  were  30  questions,  15  of  which  were  worded  to  be  answered  "yes"  and  15 
"no"  by  a  completely  positive  test  subject.  The  questionnaire  was  subjected  to  the  Stu¬ 
dent  Problem  Package  (SPP)  analysis3  to  analyze  the  items  and  also  to  determine  if  any 
classification  of  test  subjects  was  significantly  different  from  the  others.  The  results 
are  reported  in  Chapter  3  and  are  shown  in  Appendix  C. 


-M 


3D.  Harnisch  and  N.  Romy,  SPP  Student  Problem  Package  on  the  IBM-PC  (University  of 
Illinois,  1985). 
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Figure  5.  Final  test  drawing.  Specific  layers,  blocks,  colors,  and  titles  were  require¬ 
ments. 


Some  questions  that  the  test  attempted  to  answer  are: 

1.  Is  computer-based  training  effective?  Do  students  learn  as  much  as  they  would 
from  traditional  classroom  training? 

2.  Which  features  are  most  effective?  Which  are  least  effective? 

3.  How  does  perceived  competency  compare  with  actual  competency  in  the  use  of 

CAD? 

4.  How  much  time  is  required  for  computer-based  CAD  instruction? 

5.  How  effective  is  the  walk-in  training  concept? 

6.  What  factors,  if  any,  determine  success  in  computer-based  CAD  training? 

Appendix  D  is  a  blank  questionnaire.  Not  all  of  the  questions  were  answered,  but 
enough  data  were  collected  to  report  some  general  findings.  Several  new  questions  were 
raised  as  a  result  of  the  testing.  More  study  is  needed  to  answer  these  questions  and  to 
validate  the  data  from  this  test. 
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3  FINDINGS  AND  ANALYSIS 


Differences  Due  to  Test  Plan 

As  expected,  having  onsite  project  coordinators  run  the  test  produced  some  local 
differences  in  areas  that  may  have  impacted  the  results.  Among  these  differences  were 
the  following: 

•  One  site  offered  group  instruction  at  the  beginning  whereas  the  others  started 
subjects  individually. 

•  At  one  site,  the  project  coordinator  was  present  most  of  the  time;  at  the  others, 
they  were  not. 

•  One  site  had  AutoCAD  reference  manuals  nearby  and  others  had  no  manuals. 

•  Only  one  site  set  a  schedule  for  the  users. 

•  The  attitudes  of  managers  above  the  project  coordinator  ranged  from  encourag¬ 
ing  to  antagonistic. 


Sample  Demographics 

Eighty-four  subjects  participated  in  some  part  of  the  test.  Seventy-nine  completed 
the  castle  drawing  and  64  answered  the  questionnaire.  The  official  test  sites  were  repre¬ 
sented  nearly  equally  (Figure  6).  The  total  number  of  subjects  provided  by  those  sites 
was  61.  Unofficial  sites  supplied  23  more  test  subjects. 

Nineteen  percent  of  the  test  subjects  were  female.  The  total  test  sample  was  rep¬ 
resented  by  a  professional  background  as  shown  in  Figure  7,  where  "other"  signifies 
draftsmen  or  technicians  for  the  most  part.  As  may  be  observed  from  the  figure,  the 
sexes  reflected  about  the  same  proportion  within  each  professional  background  as  the 
total  sample. 


Figure  6.  Representation  from  the  three  official  test  sites. 


TOTAL  SAMPLE 


FEMALES 


ARCHITECTS 
^  18* 


Figure  7.  Representation  by  background.  Breakdown  by  sex  was  nearly  identical  to  the 
total  sample. 

With  respect  to  computer  literacy,  engineers  had  the  most  previous  experience  in 
the  sample  group,  closely  followed  by  draftsmen  and  technicians.  The  architects  had  the 
least  computer  experience  (Figure  8). 

The  average  age  of  the  test  subjects  was  36.7  years.  There  was  a  notable  differ¬ 
ence  in  average  age  of  those  with  little  computer  experience  compared  with  those  having 
extensive  experience  on  computers  (Figure  9).  The  average  age  for  subjects  with  more 
than  1  year  of  computer  experience  was  34  whereas  those  with  less  than  2  months  of 
experience  averaged  41  years  old. 

Table  1  contains  the  statistical  data  showing  differences  between  the  youngest  and 
oldest  groups.  The  level  of  significance  represents  99  percent  certainty. 

With  respect  to  experience  on  AutoCAD  and  other  CAD  systems,  only  four  subjects 
had  spent  a  great  deal  of  time  using  AutoCAD.  None  of  those  four  had  extensive  exper¬ 
ience  with  other  CAD  systems.  However,  19  percent  of  the  total  sample  claimed  exper¬ 
tise  in  another  CAD  system  (Figure  10).  The  breakdown  in  professional  background  of 
those  subjects  with  "extensive  CAD  experience"  (Figure  11)  is  quite  similar  to  that  of  the 
total  sample.  Appendix  E  shows  the  demographic  data. 
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Table  1 


Computer  Experience  by  Average  Age 


Experience 

Avg  Age  (yr) 

Std  Dev 

N 

More  than  1  yr* 

34 

8.8 

40 

2  mo.  to  1  yr 

38 

13.1 

13 

Less  than  2  mo.* 

41 

10.6 

26 

*t  =  2.896;  significant  at  the  0.01  level 
for  a  two-tailed  test. 


AutoCAD  Experience  CAD  experience  not  AutoCAD 


Figure  10.  Previous  experience  with  CAD  systems. 
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Figure  11.  Extensive  experience  with  CAD  by  background. 


Learning  Time 

The  average  time  spent  on  the  six  lessons  was  730  min  or  about  12  hr.  The  range, 
however,  was  from  46  min  to  3300  min,  with  a  standard  deviation  of  673.  Forty-six  sub¬ 
jects  kept  records  of  their  time.  One  test  site  reported  several  very  large  numbers. 
Excluding  that  site,  the  average  time  on  the  lessons  was  9.7  hr.  Students  in  the  special 
classroom  test  environment  (Chapter  2)  required  6.3  hr  to  complete  the  lessons.  The 
graph  of  scores  for  the  46  test  subjects  shows  that  82  percent  required  less  than  the 
average  time  (Figure  12). 

There  was  no  significant  relationship  (correlation  coefficient  =  0.33)  between  time 
spent  on  the  lessons  and  the  age  of  the  subject  (Figure  13).  It  is  interesting  to  note, 
though,  that  all  peaks  representing  extremely  long  periods  of  time  in  the  lessons  occurred 
in  the  over-40  group.  The  graph  of  time  with  attitude,  as  measured  by  the  questionnaire, 
also  showed  no  significant  correlation  (Figure  14). 

The  interesting  data  regarding  time  was  its  relationship  to  previous  CAD  experi¬ 
ence.  Subjects  with  experience  using  a  CAD  system  required  about  half  the  time  to 
complete  the  lessons  as  those  with  no  other  CAD  experience.  The  results  are  shown  in 
Figure  15  and  Table  2. 


Drawing  Scores 

The  drawing  scores  averaged  83  percent  with  a  standard  deviation  of  12.3.  One  of 
the  test  directions  had  proved  to  be  confusing:  the  term  "title  block"  had  a  double  mean¬ 
ing  to  many  of  the  students.  Success  on  that  item  seemed  to  be  related  to  the  subject's 
CAD  experience  and/or  to  test  coordinator  assistance.  If  that  item  were  removed  from 
the  scoring,  the  average  test  score  would  be  above  90  percent.  AutoCAD  drawings  from 
students  not  trained  by  the  computer  lessons  averaged  90  percent;  however,  there  was  no 
comparison  of  background  information  in  that  group  and  the  sample  was  very  small.  In 
the  special  classroom  test  environment,  the  average  drawing  score  was  89  percent.  The 
instructor  was  not  present  during  the  testing  period. 
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Figure  12.  Time  spent  in  the  six  AutoCAD  lessons. 
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Figure  13.  Plot  of  subject  age  and  time  spent  on  the  lessons. 
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Figure  14.  Plot  of  subject  attitude  and  time  spent  on  lessons. 
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Figure  15.  Time  spent  on  lessons  by  previous  CAD  experience  other  than  AutoCAD. 
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Table  2 


CAD  Experience  Other  Than  AutoCAD 
by  Time  Spent  on  Lessons* 


Experience 

Avg  Time  (min) 

Std  Dev 

N 

More  than  1  yr 

491 

{479}  {325} 

470.9 

8 

2  mo.  to  1  yr 

471 

202.6 

12 

None 

950 

807.1 

26 

*t  =  2.45,  significant  at  the  0.01  level  for  a  two-tailed 
test  (99  percent  certainty).  The  first  two  groups, 
representing  some  experience  in  CAD  systems  other  than 
AutoCAD,  were  grouped  for  the  test.  The  average  and 
standard  deviation  are  shown  in  brackets  { }. 


Most  of  the  factors  considered  in  this  study  had  no  significant  effect  on  the  draw¬ 
ing  scores.  The  test  sites  were  nearly  equal  in  scoring  (Figure  16). 

Architects  averaged  85  percent,  engineers  83  percent,  and  the  others  80  percent. 
The  comparisons  are  shown  in  Figure  17.  There  was  no  difference  on  the  basis  of  sex  or 
computer  experience  (Figure  18).  AutoCAD  experience  did  not  seem  to  affect  scores 
(Figure  19),  but  extensive  experience  with  another  CAD  system  did  produce  significantly 
higher  scores  (Figure  20  and  Table  3).  The  plots  of  score  by  age,  time,  and  attitude  as 
measured  by  the  questionnaire  show  no  significant  correlations  (Figures  21  through  23). 
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Figure  16.  Drawing  scores  by  site.  The  fourth  column  represents  the  largest  unofficial 
test  site. 
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Drawing  scores  by  CAD  experience  on  programs  other  than  AutoCAD 


Table  3 


CAD  Experience  Other  Than  AutoCAD 
by  Drawing  Score* 


♦The  last  two  groups,  representing  less  than  expert 
experience  in  CAD  systems  other  than  AutoCAD, 
were  grouped,  t  =  2.349;  significant  at  the  0.025 
level  (97.5  percent  certainty  for  a  two-tailed  test); 
78  df.  t  statistic  needs  to  be  2.37  for  0.01  or 
99  percent  certainty. 
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Figure  21.  Plot  of  subject  age  by  drawing  score. 
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Figure  22.  Plot  of  time  on  lessons  by  drawing  score. 
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Figure  23.  Plot  of  attitude  by  drawing  score. 


Results  of  the  Questionnaire 

Some  differences  were  observed  between  different  groups  and  their  questionnaire 
responses.  Fifty-five  percent  of  the  inexperienced  computer  users  felt  that  they  learned 
better  from  the  computer  lessons;  60  percent  of  the  nonarchitects  felt  that  way.  Forty- 
three  percent  of  the  architects  said  that  they  did  not  understand  AutoCAD,  with  only  15 
percent  of  the  other  subjects  reporting  a  poor  understanding  of  AutoCAD.  Sixty-three 
percent  of  the  architects  responding  listed  themselves  as  inexperienced  with  computers. 
This  compares  with  20  percent  for  the  entire  sample.  None  of  the  architects  said  they 
could  teach  AutoCAD;  50  percent  of  the  others  thought  that  they  could.  The  subjects 
who  had  no  previous  CAD  experience  stated  that  they  did  not  understand  AutoCAD  more 
than  three  times  as  often  as  those  who  had  experience  in  CAD  systems  other  than 
AutoCAD  (Figure  24). 

Results  of  the  questionnaire  showed  no  significant  differences  between  categories 
of  test  subjects  with  respect  to  age,  sex,  and  background.  Subjects  were  ranked  accord¬ 
ing  to  their  total  favorable  responses.  This  analysis  also  ranked  the  test  items  and 
reported  the  percentages  of  subjects  who  agreed  with  the  item  statement.  All  state¬ 
ments  were  worded  positively  for  this  purpose,  though  half  were  worded  negatively  in  the 
actual  questionnaire  (see  Appendix  D).  Agreement  with  the  statements  indicated  a 
favorable  attitude  toward  the  computer-based  training  or  some  component  related  to  it. 
Notice  that  all  of  the  statements  in  Table  4  have  more  than  75  percent  of  the  users  in 
agreement.  Lines  are  drawn  in  Table  5  to  indicate  the  cutoff  for  50  and  25  percent 
agreement.  Tables  4  and  5  list  the  questionnaire  items  in  descending  order  of  agreement. 

The  program  used  to  analyze  the  questionnaire  (the  SPP  package  noted  in  Chapter 
2)  provided  information  on  test  reliability  and  item  analysis.  The  reliability  coefficient 
(Cronbach's  alpha)  was  0.82,  which  was  in  the  acceptable  range  for  predicting  that  the 
questionnaire  would  give  similar  results  in  future  testing.  A  modified  caution  index  was 
given  for  both  the  test  subjects  and  items  on  the  questionnaire.  This  index  reports  sub¬ 
jects  who  answer  the  questions  in  an  unusual  pattern  compared  with  the  rest  of  the  sam¬ 
ple  and  also  items  for  which  responses  deviate  from  the  majority  of  the  subjects.  In  the 
case  of  subjects,  the  index  would  indicate  that  a  person  did  not  conform  with  the  attitude 
expressed  by  the  rest  of  the  group.  In  the  case  of  items,  it  would  identify  those  that  did 
not  conform  to  the  favorable-unfavorable  scale  of  the  test.  Items  that  did  not  depend  on 
a  pro  or  con  attitude  were  those  such  as  nos.  23  and  24: 

•  The  computer  helps  me  consider  different  possibilities  in  my  design. 

•  The  reminder  pages  were  annoying. 

These  statements  had  high  caution  indices  and  can  be  interpreted  as  reflecting  an  atti¬ 
tude  on  a  scale  other  than  favorable-unfavorable.  In  general,  items  or  subjects  having 
high  caution  indices  that  fall  in  the  middle  of  the  scale  carry  more  information.  The  cri¬ 
tical  value  used  for  the  modified  caution  index  was  0.30.  Appendix  C,  the  Student  Prob¬ 
lem  (SP)  chart,  reports  further  information  about  items  and  students. 


Analysis  of  Findi.igs 

The  test  scores  were  uniformly  good,  even  though  the  time  spent  and  the  subjects' 
backgrounds  varied  widely.  This  finding  suggests  that  the  professionals  involved  in  this 
test  sample  knew  enough  about  their  own  state  of  knowledge  to  spend  extra  time  on 
material  they  did  not  fully  understand.  Such  audience  control  could  probably  be  included 


Figure  24.  Understanding  of  AutoCAD  based  on  CAD  experience. 


in  many  program  options  with  confidence  that  users  will  exercise  that  control  wisely.  In 
some  studies  of  computer-aided  instruction  with  less  sophisticated  users  than  tested 
here,  this  statement  has  not  proven  true.4 

One  condition  that  became  apparent  early  in  the  testing  was  that  students  with 
certain  learning  styles  had  more  difficulty  with  the  computer-based  tutorial  than 
others.  Three  types  of  learning  styles  were  identified:  ’’discovery"  or  those  who  prefer 
to  explore  the  software  undirected,  "structured"  or  those  who  prefer  a  set  plan  for  each 
learning  objective,  and  "guided  discovery"  or  those  who  like  some  direction  but  may 
choose  to  explore  at  times.  It  was  noted  in  a  previous  report5  that  students  with  a 
"guided  discovery"  mode  of  learning  make  better  use  of  context-sensitive  embedded 
help.  In  this  test,  such  learners  also  found  the  tutorials  more  helpful.  "Discovery"  learn¬ 
ers  often  left  the  lesson  format  to  experiment  on  their  own.  While  they  did  learn  a  great 
deal,  they  may  not  have  used  their  time  as  efficiently.  Their  experimentation  might  well 
have  been  postponed  until  they  had  at  least  skimmed  the  lessons.  Users  with  a  "struc¬ 
tured"  learning  style  were  often  frustrated  by  the  freedom  to  explore  and  the  lack  of  a 
detailed  course  of  study  and  user  manual. 


4E.  Steinberg,  "Review  of  Student  Control  in  Computer-Assisted  Instruction,"  Journal  of 
Computer-Based  Instruction,  Vol  3  (1977),  pp  84-90. 

5M.  Stoddard,  Learning  Styles  and  Embedded  Training:  A  Case  Study ,  Government 
Report  No.  ED-274  305  (Department  of  Energy,  Los  Alamos  National  Laboratory,  1985). 
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Table  4 


1 

■ 

■ 

■ 

i 

i 
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Items  With  75  Percent  or  Greater  Agreement 


Agree  Disagree 

Questionnaire  Item  (%)  (%) 


Computers  are  helpful  in  the  design  process.  100  0 

I  liked  being  able  to  page  back  and  use  an  index.  99  1 

Trying  the  AutoCAD  commands  helped  me  learn.  99  1 

I  like  working  with  computers.  96  4 

I  believe  that  I  could  use  AutoCAD  in  my  work  92  8 

projects. 

I  expected  to  like  doing  the  computerized  92  8 

instruction. 

The  use  of  various  colors  for  text  (not  the  drawings)  92  8 

throughout  the  lessons  was  useful. 

The  time  I  used  for  lessons  was  well  spent.  89  11 

I  like  AutoCAD.  88  12 

I  like  computer-based  instruction.  87  13 

The  lessons  were  not  too  difficult  for  me.  87  13 

I  usually  knew  when  I  did  things  wrong  in  the  86  14 

lessons. 

The  computer  helped  me  consider  different  83  17 

possibilities  in  my  design. 

I  understand  much  about  AutoCAD.  83  17 

Most  of  the  lessons  were  not  too  long  for  a  single  75  25 

session. 

The  computer  allowed  me  to  express  my  design  75  25 

ideas  as  well  as  l  can  manually. 
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Table  5 


Items  With  Less  Than  75  Percent  Agreement 


Questionnaire  Item 

Agree 

<%) 

Disagree 

<%) 

The  first  lesson  was  easy  enough  that  I  didn't  feel 
frustrated. 

73 

27 

I  feel  generally  competent  with  the  AutoCAD 
concepts  covered  in  the  computerized  lesson. 

73 

27 

The  reminder  pages  were  not  annoying. 

72 

28 

I  like  walk-in  training  better  than  scheduled  classes 
and  labs. 

57 

43 

I  don't  need  a  time  schedule  to  keep  myself  working 
on  the  computer-based  instruction. 

56 

44 

50% - 

Having  previous  computer  experience  is  unneces-  48  52 

sary  before  using  the  embedded  AutoCAD  instruction. 

I  feel  capable  of  teaching  AutoCAD  to  others.  46  54 

Having  previous  experience  in  drawing  and/or  45  55 

drafting  is  unnecessary  before  attempting  the 
AutoCAD  instruction. 

The  lessons  helped  me  learn  more  quickly  than  I  do  41  59 

from  classrooms  or  textbooks. 

A  teacher  need  not  be  present  during  the  computer-  40  60 

ized  lessons. 

I  did  not  need  more  feedback  from  the  computer  36  64 

program. 

I  did  not  have  trouble  following  the  directions  in  the  36  64 

lessons. 

25% - 

I  do  not  need  more  coursework  to  use  AutoCAD  20  80 

productively. 


A  manual  covering  the  lessons  is  unnecessary.  18  82 
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The  architects  showed  a  tendency  to  differ  in  their  acceptance  of  the  program; 
however,  too  few  were  tested  to  make  a  sound  judgment.  At  one  test  site,  there  could 
have  been  many  more  architects  involved  in  the  testing,  but  they  were  not  interested  in 
doing  the  lessons.  Some  of  them  had  had  experience  with  another  CAO  system  and 
resisted  learning  AutoCAD.  At  a  second  test  site,  where  there  were  no  CAD  experts  on 
any  system,  AutoCAD  was  accepted  quickly.  The  coordinator  reporttd  that  persons  were 
doing  production  drawing  shortly  after  completing  the  lessons.  Some  individuals  at  the 
test  sites  indicated  that  they  did  not  want  to  learn  a  different  CAD  system.  It  should  be 
noted  that  finding  a  way  to  combat  bias  against  learning  a  new  CAD  system  is  an  impor¬ 
tant  issue,  especially  since  the  data  show  that  those  who  do  learn  a  second  program  score 
higher,  require  less  learning  time,  and  feel  that  they  understand  CAD  better. 

The  potential  benefit  of  learning  a  second  CAD  system  is  extremely  important  for 
further  study.  The  observed  bias  seemed  to  be  centered  on  the  hesitancy  to  learn  a  dif¬ 
ferent  set  of  procedures  in  a  new  system  because  the  individual  would  be  threatened  with 
memory  overload.  However,  the  evidence  suggests  that  conceptual  understanding  causes 
the  faster  and  better  learning;  when  the  concepts  are  established,  the  procedures  are  no 
longer  a  problem.  For  example,  complex  entities  can  be  manipulated  in  a  CAD  system  by* 
following  prompts  without  grasping  the  essential  "conceptual"  qualities  of  the  feature. 
However,  when  that  feature  is  discovered  in  another  CAD  system,  its  characteristics 
become  meaningful  and  the  designer  imagines  ways  to  explore  its  possibilities.  The  pro¬ 
cess  of  concept  development  is  thus  a  matter  of  hypothesis  formation  and  testing.  Em¬ 
bedded  instruction  offers  an  especially  useful  environment  for  that  kind  of  testing. 

Responses  from  the  test  subjects  indicated  that  most  preferred  learning  about 
computer  programs  from  interacting  with  a  computer  rather  than  studying  a  book.  While 
help  is  necessary  for  a  few  individuals,  most  become  independent  very  quickly.  One  at¬ 
tractive  aspect  of  training  for  adults  is  that  the  instructional  course  need  not  cover 
every  detail  of  the  entire  subject.  Once  users  are  comfortable  with  the  concepts  and 
understand  the  scope  of  the  program,  application  to  specific  problems  is  possible.  At 
that  time,  reference  manuals  and  other  users  proficient  in  the  same  specialty  can  serve 
as  efficient  instructional  guides.  The  tests  have  shown  that  embedded  computer-based 
training  can  present  basic  techniques  and  foster  the  development  of  a  conceptual 
approach  to  problem-solving.  However,  ECS,  the  company  marketing  the  program,  will 
supply  it  with  documentation  since  some  users  feel  more  comfortable  having  a  hard-copy 
manual  nearby. 


Future  Research  Direction 
Revisions 

The  test  results  proved  valuable  for  revising  the  embedded  lessons.  A  manual  was 
prepared  and  directions  for  practice  have  been  color-coded  and  reworded  based  on  sug¬ 
gestions  from  the  users.  Increased  and  expanded  optional  feedback  messages  (including 
reminders)  are  scheduled  for  a  new  version  as  are  advanced  modules.  The  walk-in  train¬ 
ing  concept  needs  further  study  and  specific  directions  for  implementation.  Future  tests 
include  case  studies  at  the  test  sites  to  explore  ways  of  adapting  to  learning  styles  and 
past  experience. 


31 


New  Programs 

As  described  above,  the  tests  indicated  that  those  who  know  two  or  more  programs 
have  a  better  understanding  of  CAD  concepts.  For  designers  to  make  the  computer  a 
part  of  their  design  process,  an  expert  level  of  understanding  of  CAD  is  needed.  Learn¬ 
ing  a  second  system  may  be  especially  valuable  for  that  reason.  In  addition,  there  are 
many  other  advantages  to  learning  a  second  system,  including  cost,  standards,  and 
ensured  compatibility  with  existing  software  or  drawings. 

The  results  concerning  bias  against  learning  a  second  CAD  system  have  prompted  a 
new  line  of  research.  Tutoring  software  is  being  designed  and  implemented  for  a  second 
CAD  system,  MicroStation.  Using  an  approach  suggested  elsewhere,6  the  new  instruc¬ 
tional  program  will  offer  special  context-sensitive  help  to  users  who  indicate  that  they 
already  know  AutoCAD.  This  help  is  likely  to  be  applicable  to  a  large  percentage  of 
USACE  personnel.  Figures  25  through  27  show  examples  of  how  that  feature  will  work. 
Users  will  be  told  that  such  help  is  available.  The  idea  is  to  make  users  feel  that  the  new 
lessons  will  build  upon  their  past  knowledge  rather  than  treat  them  as  beginners.  In 
interviews  with  those  resistant  to  learning  new  software,  users  have  said,  "I  don't  have 
the  time  to  do  that”  and  "I  am  just  getting  really  good  at  the  system  I  have."  They  ex¬ 
pect  the  second  system  to  be  totally  different  and  as  difficult  to  learn  as  the  first. 
Actually,  in  many  cases,  knowledge  of  another  CAD  system  is  helpful,  as  the  tests  have 
shown,  but  in  a  few  cases  those  who  know  AutoCAD  will  have  to  "unlearn"  that  program 
and  may  need  extra  help.  The  lesson  design  takes  this  possibility  into  account.  The 
approach  to  meeting  individual  needs  will  be  evaluated  carefully  to  determine  if  more 
complex,  extensive  programs  of  diagnostics  and  adaptive  branching  should  be  attempted. 

In  response  to  requests  for  more  feedback  and  giving  the  user  more  control  over  the 
program,  the  new  program  will  offer  an  optional  exercise  check  to  monitor  the  drawing 
file  and  offer  very  specific  information  to  the  user.  This  program  will  diagnose  user 
additions  to  the  drawing  data  base.  If  they  satisfy  the  requirements  of  the  task  stated  in 
the  exercise,  the  user  will  receive  the  "OK"  feedback.  If  the  requirements  are  not  met, 
the  program  will  attempt  to  determine  how  the  problem  occurred  and  offer  corrective 
feedback  if  possible.  For  unpredicted  errors,  the  exercise  should  be  repeated  and  the 
user  encouraged  to  try  again.  Also,  there  should  be  a  reminder  to  try  the  exercise  if  no 
changes  have  been  made  in  the  drawing  file.  The  expert  critic  will  be  available  only  if 
the  user  calls  the  "check"  function.  If  the  student  is  merely  browsing  or  reviewing,  he  or 
she  may  not  choose  to  be  interrupted  by  the  computer  system.  Figure  28  shows  an 
example  of  this  diagnostic  and  prescriptive  process. 

These  program  improvements  are  intended  to  help  overcome  bias  against  learning 
new  CAD  software.  Giving  the  learner  more  options  and  more  precise  feedback  should 
also  make  the  process  more  efficient. 


6D.  Brown,  From  Pascal  to  C  (Wadsworth  Publishing,  1985). 
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SHAPE  PLACEMENT:  4of  5 


PLACE  BLOCK  pi  ices  a  rectangular  shape  when  two  diagonal  corner) 
are  selected.  The  block  will  be  placed  orthogonal  to  the  view. 


SHAPE 


PLACE  BLOCK  ^ 

□ 

0 

0 

Try  PLACE  BLOCK  in  the 

space  below. 


X- 

HELP 


Figure  25.  Page  requiring  help.  This  screen  may  introduce  problems  for  users  who 
know  AutoCAD.  The  drawing  shows  the  picture  found  on  the  MicroStation 
menu  template. 


r 


A 


Do  not  become  confused  with  the  BLOCK  in  AutoCAD. 

The  Microstation  BLOCK  is  a  special  case  of  the  shape. 
The  AutoCAD  BLOCK  is  very  much  like  CELL  in 
Microstation. 


V 


Figure  26.  Help  for  users  experienced  with  AutoCAD.  AutoCAD  users  may  have  a 
problem  with  terminology  here.  They  will,  however,  have  a  conceptual 
understanding  of  complex  entities,  once  the  naming  is  explained  that  the 
new  CAD  learner  lacks. 


Figure  27.  Help  for  MicroStation  users.  New  MicroStation  users  receive  the  message 
above.  They  should  not  find  the  BLOCK  confusing;  however,  unless  they 
have  some  previous  CAD  experience,  they  probably  have  no  deeper  under¬ 
standing  about  complex  entities. 


Figure  28.  Check  and  feedback  option.  The  system  will  check  for  the  required 
circle.  One  of  the  following  concept-oriented  messages  would  be  given  to 
the  user:  (1)  "Good  work,  your  circle  is  correct.  Computers  are  very  good 
at  this  sort  of  thing,  which  is  time-consuming  for  humans";  (2)  "The  circle 
is  not  quite  correct.  Usually  we  know  the  radius  or  the  center  of  a  circle 
but  not  that  is  not  the  case  here.  Try  PLACE  CIRCLE  EDGE";  (3)  "I  can't 
find  your  circle.  Try  again  and  then  check." 
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4  CONCLUSIONS  AND  RECOMMENDATIONS 


A  computer-based  educational  system  has  been  developed  and  tested  in  the  field. 
The  system  uses  embedded  technology,  in  which  the  instructional  program  is  contained 
within  the  actual  software  to  be  learned.  Users  thus  can  work  through  a  training  pro¬ 
gram  at  their  own  pace  and  according  to  their  individual  needs.  In  addition,  users  are  af¬ 
forded  hands-on  experience  with  the  system  they  will  actually  be  using  in  daily  opera¬ 
tions,  making  the  training  more  realistic  than  in  other  types  of  instruction  (e.g.,  an  off¬ 
site  course  on  a  dissimilar  system).  Embedded  training  programs  also  offer  the  advan¬ 
tage  of  being  more  cost-effective  than  other  options.  Estimates  based  on  the  cost  of 
producing  the  embedded  instruction  compared  with  classroom  training  offered  by  a 
vendor,  including  the  salaries  of  the  professionals  being  trained  and  travel/per  diem 
costs,  show  an  8:1  advantage  in  favor  of  CBI. 

To  develop  the  embedded  program,  USA-CERL  first  studied  A/Es  who  were  learn¬ 
ing  to  use  a  CAD  system  and  identified  educational  elements  that  would  be  needed  to  en¬ 
sure  a  successful  program.  A  prototype  program  for  learning  CAD  was  then  produced 
and  officially  tested  at  three  District/Division  Offices  plus  some  volunteer  sites. 

The  results  showed  that  A/Es  benefit  from  online  instruction  in  the  CAD  program 
they  wish  to  learn.  The  wide  variability  in  the  time  spent  studying  the  lessons  and  in  the 
learning  strategies  used  by  the  students  underscore  the  objectives  of  embedded  instruc¬ 
tion-self-paced,  self-serving  education.  A  questionnaire  administered  to  test  partici¬ 
pants  revealed  a  preference  for  this  type  of  instruction  over  traditional  classroom 
methods. 

The  professionals  who  served  as  test  subjects  showed  a  clear  understanding  of  their 
own  needs  and  state  of  knowledge,  which  optimized  their  time  spent  in  the  instructional 
program.  This  finding  suggests  that  tutorials  for  professionals  at  different  levels  could 
be  structured  to  be  "intelligent"  in  providing  only  the  degree  of  help  needed  by  the 
learner.  In  other  words,  educational  software  for  such  an  audience  can  be  largely 
learner-controlled. 

A  major  factor  influencing  motivation  to  use  the  embedded  program  was  the  user's 
reluctance  to  learn  a  second  CAD  system.  A/Es  who  may  have  had  a  difficult  time 
learning  one  CAD  program  expected  the  training  to  be  complicated  and  not  worth  their 
time.  In  fact,  the  research  has  shown  that  users  who  know  more  than  one  CAD  program 
have  a  much  easier  time  learning  another  one  because  they  use  conceptual  knowledge 
rather  than  a  procedural  approach.  This  area  merits  further  study  to  find  ways  of 
removing  the  bias  against  learning  new  programs. 

The  product  of  this  research  was  the  revised  and  enhanced  software  instruction 
package  which  was  turned  over  to  Electronic  Courseware  Systems,  Inc.,  through  a 
CRDA.  It  is  being  marketed  nationwide  as  the  "Teaching  Assistant  for  AutoCAD." 

It  is  recommended  that  USA-CERL  continue  to  study  embedded  instruction  pro¬ 
grams  to  identify  other  potential  applications  for  USACE.  In  addition,  the  research 
should  be  directed  toward  finding  ways  to  increase  users'  willingness  to  approach  new 
CAD  programs  as  a  way  of  becoming  proficient  in  the  CAD  concept.  This  increased  pro¬ 
ficiency  will  enable  A/Es  to  use  CAD  in  the  early  stages  of  design,  become  more  produc¬ 
tive  in  daily  drafting  work,  adapt  more  easily  to  new  software  programs  introduced  into 
the  work  environment,  and  gain  greater  confidence  and  comfort  with  automated  drafting. 
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APPENDIX  A: 


FORMS  FOR  SUBJECT  INFORMATION 


Personal  Record 


Name _ El* _ Phone  ID.  No, 


ID.  No. 

SEX 

AGE 

Background: 

Architect 
Engineer 
oraD 
Other _ 

Student  Q 

Year  completed  education  _ 

Experience: 

Computers:  more  than  1  year 

2  months  to  1  year 
less  than  2  months 


AutoCAD:  more  than  6  months 

less  than  6  months 
none 


B 


Engineering  discipline 


Computer  Graphics  Programs  (other  than  AutoCAD) 

extensive  s— 
little  ^ 

none  P" 


Figure  Al.  Personal  record  form. 
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ID.  NO. 


TIME  AND  COMMENTS 


APPENDIX  B: 


FINAL  DRAWING  TEST 


AutoCAD  Final  Evaluation  Exorcise 

READ  THIS  COMPLETELY  BEFORE  YOU  BEGIN  YOUR  DRAWING.  This  is  a  final 
excercise  to  see  what  you  have  learned  about  AutoCAD.  The  computer 
should  keep  a  record  of  the  time  you  spend  on  the  drawing.  Just  in  case 
something  happens,  I  would  like  you  to  enter  your  total  time  in  the  title 
block  just  before  you  save  the  drawing.  Please  save  the  drawing  to  the 
floppy  disk  drive  (  A:  ).  If  you  don't  understand  about  that,  see  your  test 
coordinator. 

You  may  follow  the  suggestions  below  to  obtain  the  drawing.  The  objec¬ 
tive  is  to  make  an  exact  copy  of  the  castle  drawing  and  a  title  block  that 
contains  the  information  listed.  Your  drawing  should  look  like  EXAMPLE  1. 
EXAMPLE  2  is  included  to  show  you  exact  locations  and  a  suggested  grid. 
You  don't  need  to  follow  the  steps  if  you  would  rather  do  it  your  own  way, 
but  be  sure  that  your  final  result  is  the  same.  Suggestions  for  specific 
commands  are  only  meant  to  guide  you. 

1.  From  the  AutoCAD  main  menu  begin  a  new  drawing.  Name  it  “A:TEST***” 

where  . represents  your  three  digit  ID  number.  Be  sure  that 

you  have  a  formatted  disk  in  drive  A. 

2.  Set  the  drawing  parameters: 

UNITS:  Architectural 

LIMITS:  lower  left  (  -10’, -10')  upper  right  (94',  54’) 

(Your  screen  may  be  a  little  larger  in  one  of  the  dimensions,  but  be 
sure  that  the  X  and  Y  are  at  least  as  large  as  the  above.) 

Be  sure  to  ZOOM.  ALL  after  you  set  the  limits. 

Set  the  GRID:  and  SNAP:  to  any  values  that  seem  to  be  helpful. 

Don't  forget  that  COORDS:  and  ORTHO:  may  be  turned  on. 

BLIPS:  may  oe  turned  off  if  you  like. 

You  may  want  to  use  OSNAP:  for  accuracy. 

3.  Create  a  layer  named  CASTLE  and  color  it  red.  Draw  the  castle  on 

layer  CASTLE  to  the  exact  dimensions  of  EXAMPLE  1.  You  will  need  to 
use  LINE:,  ARC:  and  CIRCLE  to  accomplish  the  drawing.  You  may  type 
numeric  input  or  use  the  drawing  tools  for  accuracy.  Don't  forget 
that  you  can  use  COPY:,  MIRROR:  and  ARRAY:  to  save  time.  The  points 
on  the  example  show  you  centers  for  the  door  arc  and  the  circle.  The 
small  circles  have  a  diameter  of  1'.  To  duplicate  the  circles,  try 
ARRAY:  POLAR  (9  items  fill  180  degrees)  and  ARRAY:  RECTANGULAR. 
The  windows  measure  4'  X  8'.  Also,  if  you  need  to  edit,  don't  forget 
MOVE:,  CHANGE:,  and  BREAK. 


4.  Create  a  layer  named  TITLE,  which  you  will  leave  white.  Make  a  block 

called  TITLE  which  contains  the  information  shown  on  the  drawing. 
You  may  find  it  helpful  to  ZOOM:  in  on  the  title  block  for  the  detail 
work.  The  trace  width  for  the  border  is  6*.  If  you  are  in  snap  mode, 
you  will  probably  need  to  turn  it  off  to  complete  the  border.  Text 
size  for  the  lettering  should  be  V.  Insert  the  TITLE  block  on  layer 
TITLE  and  enter  the  appropriate  information. 

5.  For  an  added  challenge  (optional),  create  a  layer  named  dimension  and 

enter  3  or  4  dimensions  that  seem  important  to  you.  Remember  that 
you  may  need  to  change  the  scale  of  the  dimensioning  text  and 
arrows  with  the  DIMSCALE  variable  (Try  a  value  of  60.00). 

The  last  thing  you  should  do  before  you  save  your  work  is  enter  the  total 
time  you  spent  on  the  exercise  in  the  blank  on  your  title  block. 

When  you  are  finished  with  the  drawing,  type  SAVE.  The  computer  will 
ask:  file  name  <test***>. 

If  the  name  is  correct,  press  Enter  and  your  file  will  be  saved. 

Make  corrections  in  the  name  if  necessary. 

After  the  file  is  saved  to  the  floppy  disk,  give  the  disk  to  the  coordinator. 
You  may  then  QUIT  the  file  and  exit  AutoCAD. 

Congratulations! 
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APPENDIX  C: 


STUDENT  PROBLEM  CHART 


The  SP  chart  (Figure  Cl)  shows  individual  test  subjects'  responses  on  each  test 
item.  Responses  that  agree  with  the  "pro"  attitude  toward  the  use  of  computers,  com¬ 
puter-based  training,  and  the  CAD  system  are  represented  by  an  asterisk  (*)  and  those 
disagreeing  are  represented  by  a  digit.  The  subjects  are  listed  vertically  with  their  ID 
numbers  at  the  beginning  of  each  row  and  the  items  are  listed  horizontally  with  item 
numbers  at  the  top  and  bottom  of  the  column.  The  points  at  which  each  subject's 
responses  would  be  expected  to  be  in  agreement,  based  on  his  or  her  score,  are  plotted  by 
the  solid  line.  The  points  at  which  each  item  would  be  expected  to  be  answered  with 
agreement,  based  on  the  total  agreement  score  for  the  item,  are  plotted  by  the  dotted 
line.  All  subject  responses  to  the  left  of  the  solid  line  would  be  expected  to  be  repre¬ 
sented  by  asterisks  and  those  to  the  right  by  digits.  All  item  responses  above  the  dotted 
line  would  be  expected  to  be  represented  by  asterisks  and  those  below  by  digits.  The 
chart  produced  in  this  study  was  very  close  to  expectations  for  both  subjects  and  items. 
The  results  show  that  the  subject  group  was  fairly  homogenous  and  that  the  items  mea¬ 
sured  the  expected  traits. 
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Figure  Cl.  SP  Chart. 


APPENDIX  D: 


QUESTIONNAIRE 


ID  NUMBER 


QUESTIONNAIRE 


RESPOND  TO  THE  FOLLOWING  STATEMENTS  BY 
CHECKING  THE  AGREE  OR  DISAGREE  COLUMN. 

PLEASE  DO  NOT  OMIT  ANY  ITEMS,  PICK  THE  ONE  THAT  BEST  APPLIES. 


AGREE  .  DISAGREE 


1.  I  feel  generally  competent  with  the  AutoCAD 
concepts  covered  in  the  computerized  lessons. 

2.  I  like  AutoCAD. 

3.  Having  previous  experience  in  drawing  and/or 
drafting  is  necessary  before  attempting  the 
AutoCAD  instruction. 

4.  A  manual  covering  the  lessons  is  necessary. 

5.  The  lessons  helped  me  learn  more  quickly  than  I  do 
from  classrooms  or  textbooks. 

6.  In  order  to  use  AutoCAD  productively,  I  need  more 
coursework. 

7.  The  computer  won’t  allow  me  to  express  my  design 
ideas  as  well  as  I  can  manually. 

8.  I  believe  that  I  could  use  AutoCAD  in  my  work 
projects. 

9.  I  had  trouble  following  the  directions  in  the  lessons. 

10.  I  feel  capable  of  teaching  AutoCAD  to  others. 

11.  The  first  lesson  was  easy  enough  that  I  didn't  feel 
frustrated. 

12.  I  like  working  with  computers. 

13.  Trying  the  AutoCAD  commands  helped  me  learn. 

14.  Most  of  the  lessons  were  too  long  for  a  single 
session. 

15.  The  lessons  were  too  difficult  forme. 
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AGREE  DISAGREE 

16.  Computers  are  not  helpful  in  the  design  process. 

17.  The  time  I  used  for  the  lessons  was  well  spent. 

18.  I  needed  more  feedback  from  the  computer  program. 

19.  A  teacher  should  be  present  during  the  computerized 
lessons. 

20.  The  use  of  various  colors  for  text  (not  the  drawings) 
throughout  the  lessons  was  useful. 

21.  I  need  a  time  schedule  to  keep  myself  working  on  the 
computer-based  instruction. 

22.  I  liked  being  able  to  page  back  and  use  an  index. 

23.  The  computer  helps  me  consider  different 
possibilities  in  my  design. 

24.  The  reminder  pages  were  annoying. 

25.  Having  previous  computer  experience  is  necessary 
before  using  the  embedded  AutoCAD  instruction. 

26.  I  expected  to  like  doing  the  computerized 
instruction. 

27.  I  don't  understand  much  about  AutoCAD. 

28.  I  usually  knew  when  I  did  things  wrong  in  the 
lessons. 

29.  I  like  walk-in  training  better  than  scheduled  classes 
and  labs. 

30.  I  don’t  like  computer-based  instruction. 

31.  I  prefer  micro  CADD  to  mainframe  CADD. 

32.  I  would  like  to  see  more  advanced  lessons  made 

available.  | 

Please  use  the  back  of  this  sheet  for  additional  comments  or  suggestions  about  the 
computer-based  instruction. 
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Table  E2 


Results  of  Questions  1  Through  10 
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Table  E3 


Results  of  Questions  11  Through  20 


■■ogMnHMnMBHHnaaMHHHBHHHnHHnaMgg 


BS^^BSB&8^S8Sffi&E8SSBSS8Bjfl3S^BSS58SSS&S8BSSSSS 


!3555SS5i 


mm&za 


47 


Table  E4 


Results  of  Questions  21  Through  30 


USA-CERL  DISTRIBUTION 


Chief  of  Engineers 
ATTN:  CEIM-SL  (2) 

ATTN:  CE1M-S 
ATTN:  CECC-P 
ATTN:  CECW 
ATTN:  CECW-0 
ATTN:  CECW-P 
ATTN:  CECW-RR 
ATTN:  CEEC 
ATTN:  CEEC-C 
ATTN:  CEEC-E 
ATTN:  CEEC-ED 
ATTN:  CEEC-ES 
ATTN:  CEEC-EC 
ATTN:  CERD 
ATTN:  CERD-C 
ATTN:  CERD-M 
ATTN:  CERM 
ATTN:  DAEN-ECE-M 
ATTN:  DAEN-ZCE 
ATTN:  DAEN-ZCI 
ATTN:  DAEN-ZCM 
ATTN:  DAEN-ZCZ 

USAEHSC,  ATTN:  Library  22060 
ATTN:  CEHSC-FM 
ATTN:  CEHSC-FB-S 

US  Army  Engineer  Districts 
Alaska  99S06 

ATTN:  CENPA-EN-BB-A 
Charleston  29402 
ATTN:  CESAC 
ATTN:  CESAC-EN-DF 
ATTN:  CESAC-EN-DA 
Fort  Worth  78107 

ATTN:  CESWF-ED-DA 
Kansas  City  64106 

ATTN:  CEMRKED-P 
Memphis  9*103 
ATTN:  CELMM 
ATTN:  CELMN-IM-R 
Mobile  36*28 
ATTN:  CESAM 
Nashville  37202 
ATTN:  CEORN 
Norfolk  23510 

ATTN:  CENAD-IMO 
Philadelphia  19106 
ATTN:  CENAP 

Sacramento  95814 

ATTN:  SPKED-A  (2) 
ATTN:  CESPK 
ATTN:  CESPK-PB 
ATTN:  CESPK-CO-C 
Saint  Louis  63101 

ATTN:  CELMSED-D 
Saint  Paul  55101 
ATTN:  CENCS 
Savannah  31402  (3) 

ATTN:  CESAS-EN-DA 
Vicksburg  39180 

ATTN:  CELMK-IM-IC 
Willmington  28402 
ATTN:  CESAW 


US  Army  Engineer  Divisions 
Huntsville  35807 

ATTN:  CEHND  (2) 
ATTN:  CEHND-ED-ES 
ATTN:  Training  Center 
Missouri  River  68101  (2) 
ATTN:  MRD-ZO 
New  England  02254 
ATTN:  CENED 
North  Pacific  97208 

ATTN:  CENPD-CO-C 
ATTN:  CENPD-EN-HD 
Ohio  River  45201 

ATTN:  CEORD-ED-TS 
South  Pacific  94111 
ATTN:  CESPD-ED-T 
ATTN:  CESPD-ED 
ATTN:  CESPD-IM-R 


US  Army  Europe 

AEASN-O  DC  S/Engr  (2)  09403 
AEUES  0*0*1 
V  Corps 

ATTN:  DEH  (12) 
vn  Corps 

ATTN:  DEH  (18) 

21st  Support  Command 
ATTN:  DEH  (12) 

USA  Berlin 

ATTN:  DEH  (10) 

USASETAF 
ATTN:  DEH 

USA  Japan  (USARJ) 

ATTN:  DCSEN  96343 

ATTN:  Facilities  Engineer  9*343 

ATTN:  DEH-Okinawa  9*331 

416th  Engineer  Command  60823 
ATTN:  Facilities  Engineer 

US  Military  Academy  10966 
ATTN:  Facilities  Engineer 
ATTN:  Dept  of  Geography  4 
Computer  Science 
ATTN:  MAEN-A 

DLA  ATTN:  DLA-W1  22304 

FORSCOM  (28) 

FORSCOM  Engineer,  ATTN:  Spt  Det. 
ATTN:  Facilities  Engineer 

HSC 

Ft.  Sam  Houston  AMC  78234 
ATTN:  HSLO-F 
Fitzslmona  AMC  80045 
ATTN:  HSHG-DEH 
Walter  Reed  AMC  20307 
ATTN:  Facilities  Engineer 

INSCOM  -  Ch,  Instl.  Div. 

Arlington  Hall  Station  (4)  22212 
ATTN:  Facilities  Engineer 
Vint  Hill  Farms  Station  22186 
ATTN:  IAV-DEH 

MDW 

ATTN:  DEH  (3) 

Cameron  Station  22314 
Fort  Lesley  J.  McNair  20319 
Fort  Myer  22211 

MTMC 

ATTN:  MT-LOF  20315 
ATTN:  Facilities  Engineer  (5) 

TARCOM,  Fac.  Div.  48090 

TRADOC  (19) 

HQ,  TRADOC,  ATTN:  ATEN-DEH  23651 
ATTN:  DEH 

USA1S,  ATTN:  Facilities  Engr  (4) 

WESTCOM 

Fort  Shatter  96858 
ATTN:  DEH 
ATTN:  APEN-A 

Fort  Belvoir,  VA  22060 

ATTN:  Canadian  Liaison  Officer 
ATTN:  British  Liaison  Officer 
ATTN:  Australian  Liaison  Officer 
ATTN:  French  Liaison  Officer 
ATTN:  German  Liaison  Officer 
ATTN:  Water  Res  Support  Center 
ATTN:  Engr  Studies  Center 
ATTN:  Engr  Topographic  Lab 
ATTN:  ATZA-TE-SU 
ATTN:  STRBE-BLURE 


CECRL 

ATTN:  Library  03755 
ATTN:  CECRL-EC 

HQ,  XVIII  Airborne  Corps  and 
Ft.  Bragg  28307 

ATTN:  AFZA-FE-EE 

NAVFAC  22332 

ATTN:  Division  Offices  (11) 

ATTN:  Facilities  Engr  Cmd  (9) 

ATTN:  Code  401 

ATTN:  Naval  Public  Works  Ctr  (9) 

ATTN:  Naval  Civil  Engr  Lab  (2) 

ATTN:  Naval  Constr  Battalion  Ctr 
ATTN:  DSO  IE  (2) 

Naval  Training  Center,  FL  32813 

NCEL  93043 

ATTN:  Library  (Code  L08A) 

Engineering  Societies  Library 
New  York,  NY  10017 

SETAF  Engineer  Design  Office  09019 

U.S.  Coast  Guard  Ship  Design  Branch,  D.C.  20S93 

North  Carolina  National  Guard  28219 

HQ  3COBP8,  Fort  Hood  76544 
ATTN:  AFZF-EN 

USAFA/DFCE  80840 

WES  3*180 

ATTN:  CEWES-ME  (2) 

ATTN:  CEWES-IM-DA/CPT 
ATTN:  CEWES-IM-DS 
ATTN:  Library 


Kelley  AFB.TX  78241 
ATTN:  MAWFF 


Tyndall  AFB,  FL  32403 
ATTN:  A F ESC /SI E 

New  Cumberland  Army  Depot  17070 
ATTN:  USACE 

BLM-Chief  of  Eng.  Serv.,  CO  80225 

US  Office  of  Personnel  Management,  WA  98174 

USASETAF  APO  NY  09019 

Defense  Logistics  Agency 
ATTN:  DCASR-NY 

Dir  FD  4  CSNAV,  AZ  86011 

Los  Alamos  National  Laboratory,  NM  87545 

National  Bureau  of  Standards  20899 

Defense  Technical  Info.  Center  22314 
ATTN:  DDA  (2) 
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